Detecting Depression States Based on Sensor Data

Data Science Pipeline Tutorial

By Anastasia Kortjohn


Introduction

Depression is a mood disorder that affects more than 264 million people globally, making it one of the leading causes of disability worldwide. It's characterized by symptoms such as profound sadness, the feeling of emptiness, anxiety, sleep disturbance, as well as a general loss of initiative and interest in activities. The severity of a depression is determined by the quantity of symptoms, their seriousness and duration, as well as the consequences on social and occupational function.

One way to classify depression is unipolar and bipolar; unipolar depression refers to major depressive disorder and bipolar depression is a facet of bipolar disorder. They are both genetic mood disorders and share symptoms, but a distincion should be made between the two: bipolar depression is unique in the periodic occurrence of mania, a state associated with inflated self-esteem, impulsivity, increased activity, goal-directed actions, and reduced sleep.

Although there are known, effective treatments for mental disorders, between 76% and 85% of people in low- and middle-income countries receive no treatment for their disorder. One barrier to effective care is inaccurate assessment: in countries of all income levels, people who are depressed are often not correctly diagnosed.

How does sensor data come into play?

Actigraphs are small motion sensor detectors (accelerometers) that are encased in a unit about the size of a wristwatch, and can be worn continuously for days to months. It is well established that depression is characterized by altered motor activity, and actigraph recordings of motor activity are considered an objective method for observing depression. Despite not being exhaustively studied yet, there is an increasing awareness in the field of psychiatry on how the activity data relates to various mental health issues such as changes in mood, personality, inability to cope with daily problems, or stress and withdrawal from friends and activities.

In the following tutorial, we will walk through the Data Science Pipeline to see if depression states can be accurately predicted through the sensor data recorded by Actigraphs.


References

The Dataset

We'll be looking at Actigraphic data originally collected for a study on motor activity in schizophrenia and major depression. Actigraphs continuously record an activity count proportional to the intensity of movement in one minute intervals. The dataset consists of actigraphy data collected for the condition group (23 unipolar and and bipolar depressed patients) as well as the control group (32 non-depressed contributors). We'll be using the The Montgomery-Asberg Depression Rating Scale (MADRS) score included in the data for each participant to identify the severity of an ongoing depression. The score is based on ten items relevant for depression, which clinicians rate based on observation and conversation with the patient. The sum score (0-60) represents the severity: scores below 10 are classified as an absence of depressive symptoms, and scores above 30 indicate a severe depressive state.


References

Getting started

First let's import the libraries we'll use throughout the tutorial.

Collecting and Curating the Data

The dataset contains:

Scores Data

To get the data for the control and condition groups from scores.csv, we use pandas to read the csv file and store the data in a DataFrame. If the csv file is in another folder, the relative path must be specified, e.g. data/scores.csv.

Managing and Representing the Scores Data

Right away we can see there is a lot of missing data, mostly comprised of the depression data for the control group. We expect this to be missing for the control group, because depression data is only collected for those in the condition group; thus it's considered missing at random (MAR). The control group is also missing data for education, marriage, and work. This data could also be considered MAR, because it looks like only number, days, gender, and age were collected for the control group; therefore the data is missing because it's part of the control group. Since we'll be focusing mainly on the Actigraph data for this group, it shouldn't be a concern for the rest of the tutorial. For the condition group, there are three missing melancholia scores, and one missing education range value. It's not immediately clear which kind of missing data it is; after the following step, we'll take a closer look to see if there's any correlation between the condition group's missing data.

Notes

Since we'll first be looking at differences between the condition and control group, let's split the scores data into a control group DataFrame and a condition group DataFrame. Even though we could use the numeric indices to select a subset of the DataFrame, in the event there is a greater number of rows / the partition of indices isn't clear, it may be useful to select rows based on the column values (e.g. control or condition) as follows.

We could remove all the NaN columns for the control group since we won't use them in the analysis, but they can also be kept so the columns stay the same as the condition group DataFrame.

As mentioned previously, the condition group has missing data. It doesn't look as though it depends on any data in other columns, so it may be missing completely at random (MCAR). We can use different types of plots to display a correlation between missing values.

With only two columns containing missing data, the following missingno heatmap is not too informative, but it could be useful for larger datasets with more missing data.

Since the goal in identifying the type of missing data is to determine if and how it might affect future analyses, let's use seaborn to plot the scores we'll be using in the analysis (MADRS 1 and 2) on the X and Y axis, then see if the missing melancholia values correlate in any way. We can use the color and size to differentiate the NA values, as well as style to mark the education range values.

Other than 2 of the 3 NA points overlapping other plotploints, the figure doesn't show a particular correlation, e.g. that all the missing melancholia scores have the same MADRS 1 or 2 scores, or that all three have the same education range value. As a result, we can move forward with our analysis and treat the missing condition group data as MCAR.

Actigraph Data

Now that we've organized the scores data, we need to get all the Actigraph data. In each part, after curating the control group data, we will repeat the process for the condition group. To store the timestamps and activity data, we'll make a control list and a condition list, and fill them with DataFrames for each csv file (each participant). That way we can easily index the list by the control group. In addition, we'll map the participant number to their row counts (number of Actigraph recordings). This will help later on when selecting a range of data to plot. There is also a check in the code that will print a message if there are any missing values in the Actigraph data. Lastly, for plotting we'll also create a DataFrame with columns of just the activity data.

Control Group

Condition Group

Control Group: Activity Data Only

Condition Group: Activity Data Only

Managing and Representing the Data

As the comments in the code describe, we parse the timestamps while reading the csv so they'll be formatted as DateTime objects, which may come in handy later on in the tutorial. So far, we've prepared the data in a couple different ways for later use: a list of DataFrames, a DataFrame each for the control and condition group with the Actigraph data as columns, and we've also stored the row counts (number of Actigraph recordings) for each participant in a dictionary.

Now, let's use matplotlib to get a visual idea of the data we're working with.

We can see from these 3 samples that the Actigraph data is spread over 1 or 2 months, and there is a lot of variation in activity intensity over time. The variation might be something interesting to explore in the next section.

The subplots have their own y-axis tick values, so we need to carefully look at the range if we want to compare between the samples. From this initial glance at the data, we can see the upper range of the control group's activity reaches 2,000 (excluding outliers). Both condition 2 and 3 have a high density of points under 1000, and scattered points and spikes going up to 3000-3500. Although we can't make any assumptions about the group as a whole since we're only looking at three participants, we've seen there are certain days or weeks during the couple months recorded where the activity spikes, and the intensity value is between the 0-8000 range.

Exploratory Data Analysis and Visualization

For the first step in the exploratory data analysis, let's see if we can better understand the range and other statistics about each group using boxplots. It's important to acknowledge the size of our Actigraph datasets (up to 50,000 rows per participant) can create some difficulty when plotting. For example, we saw earlier there are frequent spikes in the activity intensity, thus we can expect there will be many outliers. This may impact the decision on whether or not to display the fliers (points past the end of the whiskers which represent the outliers). In fact, showfliers=False reduced the file size of the .ipynb notebook version of this tutorial by 15mb, so in the end the fliers had to go. For the following boxplots, we'll focus on the other information illustrated, such as medians and quartile ranges.

From the plots, we can tell the median activity for both groups is quite low; no participant seems to have a median activity much above 100. The control group generally has longer whiskers, likely because there are more outliers, which represents more spikes in activity intensity. All of the 3rd quartiles in the condition group plot and most in the control group plot are less than 500, indicating 75% of activity data is below 500.

Next let's look at the sum of activity data for each participant. This discards any consideration of variation (which we'll be looking at soon), but it will be interesting to learn if there's any correlation between total activity over time and MADRS scores. One way to approach the sums is to select a range of activity data to use; otherwise we'd be adding different amounts of activity data for each participant since there are different date ranges. For that route, we'd use the minimum number of rows we found previously, 19299. Some of the participants have 50,000 rows of activity data, so selecting 19299 of the first rows doesn't account for the case where a participant goes from one extreme to another in the middle, i.e. going from being very active to barely active. Because of that, let's normalize the activity sums instead, by summing all the activity data but dividing it by the number of rows of data, to get the average activity value for each participant. This way, we can consider the entire timeframe the participants' activity was being recorded.

Control group activity averages

Condition group activity averages

The averages for the condition group tends to be less than the control group. There also appears to be more distinct levels of variation among condition group participants (i.e. high, medium, low). A question to explore further is if these differences can be linked to different affiliation types (1: bipolar II, 2: unipolar depressive, 3: bipolar I).

In the condition group, participants 4, 7, 10, 14, and 23 have smaller activity averages; when we analyze the correlation between activity and MADRS scores in the next step, I'd expect these participants to have a higher MADRS score indicating a more severe depression (unless we will not be able to reject the null hypothesis). Since we've averaged the activity data over time, we'll also average the MADRS 1 and 2 scores (score when activity measurement started and stopped).

The participants appear to be clustered in two groups, scores of 0-20 and 22-30. Almost all those I expected to have high scores based on low activity do, except 4. This is our first visualization in the tutorial, which shows for the condition group in particular, that activity could affect the MADRS score. Looks like we could be on the right track. However, the plot is not so accurate for higher activity levels; for example, out of 9, 12, 15, and 22 (the highest average activity levels from the previous step), only one is in the lower score group like we might expect.

We know from the condition group scores that a majority of participants have affiliation type 2 (unipolar depressive). There is a pretty even split in affiliation types between the lower and higher score range. This isn't surprising given that we've taken averages of the scores and activity. In fact, it brings us to our next step: visualizing the variation and seeing if there's a correlation between the variation in activity, and the MADRS score or affiliation type.

The variance plots are not what we would hope for, since they don't show any obvious trend or cluster of data. The Loess curve, which is a local regression because the curve is weighted toward the nearest data point, shows some relation between the MADRS 1 and MADRS 2 scores, but not much else. It's clear there's no linear regression, so it was worth trying out the Loess curve option to see if we could observe some relation between the variance and MADRS score. It does appear that participants with activity in the middle variance range tend to have lower MADRS scores. One possible explanation is participants with more regulated daily/weekly activity levels experience a less severe depression.

Even though I hoped the variance would show some correlation to the MADRS scores, I think the lack of it can be explained by sleep schedules or daily routines. Activity for every participant will likely decrease during the night time hours, and thus the analysis of the variance is probably not the best way to predict MADRS scores or depression states. I think it would be interesting to explore this idea further and do something like the code below, to see if there is a correlation when plotting only certain times of the day. Since we know the circadian rythm can be affected by a depressive episode, it may also be worthwhile to plot the activity in the middle of the night.

Hypothesis Testing and Machine Learning

So far we've compared the activity data between the control and condition group, and we've also looked for correlations in activity and MADRS scores within the condition group. Through the scatterplot above, variance has proved it won't be a good predictor of depression states or MADRS scores.

The next step is to figure out if we can reject the null hypothesis: that there is no relation between activity data and depression states/MADRS scores. In the following two segments, we'll train two machine learning models to predict the affiliation type (1: bipolar II, 2: unipolar depressive, 3: bipolar I) based on the activity data. Therefore, the activity data will be the idependent variable, and the affiliation type the dependent variable. We'll be treating each participant's activity data as a feature group so that the model can learn from multiple participants' activity. We also need to select the rows of activity data to use, because the models need to be trained on the same size of data. 19299 is the minimum rows of all the condition group participants, so we use that value.

The two machine learning algorithms we'll use are Random Forest Classifier and Logistic Regression from sklearn. Random Forests is based on decision trees, which will help to classify the participant's affiliation based on the activity. Since we're working with affiliation types of 1 or 2 (there is only one 3), we can use logistic regression to see if there's a relationship between the affiliation type and activity. We'll use a test size of 7, since that's about 1/3 of the condition group.

Random Forest

The n_estimators parameter is the number of trees in the forest; generally the larger it is the better the prediction, but larger values will also cause increased computation time.

Logistic Regression

Now that we've trained the models, let's plot the confusion matrix for each so we can see how accurately the models predicted the affiliation type.

Considering we only looked at a subset of the activity data, a .7143 accuracy score is better than I was expecting. We shouldn't base our analysis on the accuracy score alone though, because it doesn't give enough information to completely evaluate the model. Looking at the heatmaps, we can see there's a true positive rate of 71% for the logistic regression classifying type 2: unipolar depressive correctly. This could indicate the activity for participants with unipolar depression is more easily distiguished than those with other affiliation types. It could also be because there are more participants with unipolar depression, there's more of a chance they'll be identified correctly based on the training data. However, if the prediction was based on number of the affiliation type, we might expect to see more false negatives as well.

I did attempt to predict MADRS scores based on activity, but it was a challenging endeavor, and had 0 or 20% accuracy only. I think we would need a machine learning algorithm that does a much more complex and in-depth evaluation of the activity data, in order to predict MADRS scores with accuracy. Because of the large variation in activity data over time, we would need a way to analyze each change in activity. Perhaps counting the number of spikes and measuring the height of those spikes could be one path forward. We might think those with a higher MADRS score will have less spikes, and for them to be of lower height because they're less active. However this may not be the case; different affiliations could impact the variation in activity. Those with bipolar depression might also experience manic episodes while activity is being measured, which would cause a quick increase in activity intensity. For that reason, a better machine learning model should probably use affiliation type and activity data as feature groups.

Huber Regressor

In a final effort to find some more meaningful results, we'll use sklearn's HuberRegressor and evaluate the model with 10-fold cross validation. The reason for using this model is it optimizes the squared loss and absolute loss, meaning that the prediction is not as influenced by outliers. Since all of the spikes in activity create a lot of outliers, using HuborRegressor for a more robust form of cross validation may allow us to get a more accurate error analysis.

Conclusion

Even though it's not too encouraging to see almost 90% error, it does illustrate how tough it is to predict something as complex as mental health. For this tutorial we are not able to reject the null hypothesis, because during the hypothesis testing we could not provide enough evidence, via the machine learning models, that it's possible to predict depression states or MADRS scores accurately based on Actigraph data. However, that also does not mean there is no relation, or that it will not be possible to predict depression states accurately based on Actigraph data. As mentioned in the introduction, Actigraph recordings are considered an objective method of observing depression, so we just need a way to build a better machine learning model.

Some of the limitations of the models during the hypothesis testing were based on data. During the exploratory data analysis and hypothesis testing, some avenues for further exploring and analyzing the data were discussed, which may help to find better feature groups and parameters to pass to the machine learning algorithms.

Overall, we were able to make distinctions between the control and condition group's activity, but not predict depression states based on that activity. We looked at the averages and variation in activity, as well as used confusion matrices (heatmaps) to display the somewhat promising predictions from the Random Forest Classifier and Logistic Regression models. Since the model is only as good as the data it's given, the next step to predicting depression states more accurately would be to explore the Actigraph data more.